In modern computer systems, large collections of data are usually organized on storage disks as files. If a large number of files exist they may be distributed over multiple disks and/or computer systems. A file system is used to control access to the files by communicating with the various disk drives. Computer programs access the files by requesting file services from one or more file systems. Some file systems also assist with error recovery functions. In most file systems, the files contain user data and metadata. Metadata is information required to manage the user data, such as names, locations, dates, file sizes, access protection, and so forth. The organization of the user data is usually managed by the client programs.
Administering a large distributed file system that serves a large number of users is a complex task. To store a larger number of files, additional disks and servers must be added. To simplify the organization of files, groups of files or “volumes” are often manually assigned to particular disks. Then, the files can by manually moved or replicated when components fill up, fail, or become throughput bound.
Joining many thousands of files distributed over many disks into a redundant array of independent disks (RAID) is one solution. Nevertheless, administration problems still arise when the system grows so large to require multiple RAIDs, and multiple server processors.
One such distributed files system has been developed by Sistina Software, Inc. and is called the Global File System (GFS). This system has been described in a number of publications including: “The Global File System” Proceedings of the Fifth NASA Goddard Conference on Mass Storage Systems by Steven R. Soltis, Thomas M. Ruwart, et al. (1996); and “A 64-bit, Shared Disk File System for Linux” Sixteenth IEEE Mass Storage Systems Symposium held jointly with the Seventh NASA Goddard Conference on Mass Storage Systems & Technologies, Mar. 15-18, 1999, San Diego, Calif. by Kenneth W. Preslan, Andrew P. Barry et. al. hereby incorporated by reference.
The Global File System (GFS) is a shared storage device, cluster file system. GFS supports multi-client reads and writes to the file system, journaling of the reads and writes, and rapid recovery from client failures. Nodes within a GFS cluster physically share the same storage by means of Fibre Channel (FC), shared SCSI devices, or network block devices. The file system is configured so that it appears to reside on each node and will synchronize a file access across the cluster. GFS is fully symmetric. In other words, all nodes are equal and there is no server which could be either a bottleneck or a single point of failure. GFS uses read and write caching while maintaining full file system semantics.
Other distributed files systems are know to those skilled in the art. For example, U.S. Pat. No. 6,173,293 issued Jan. 9, 2001 to Thekkath et al., herein incorporated by reference, discloses a file system that is distributed over a plurality of computers connected to each other by a network. The plurality of computers execute user programs, and the user programs access files stored on a plurality of physical disks connected to the plurality of computers. The file system includes a plurality of file servers executing on the plurality of computers as a single distributed file server layer. In addition, the file system includes a plurality of disk servers executing on the plurality of computers as a single distributed disk server layer, and a plurality of lock servers executing on the plurality of computers as a single distributed lock server to coordinate the operation of the distributed file and disk server layers so that the user programs can coherently access the files on the plurality of physical disks. The plurality of file servers executes independently on a different one of the plurality of computers, and the plurality of file servers communicate only with plurality of disk servers and the plurality of lock servers, and not with each other. Furthermore, the disk server layer organizes the plurality of physical disks as a single virtual disk having a single address space for the files.